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Executive Summary 


The goal of the User-Centered Digital Library Project, conducted by the National Center for 
Accessible Media (NCAM) at WGBH, was to adapt the TEACHERS’ DOMAIN online digital library to 
enable teachers and students with disabilities to more readily use the resources in science class- 
rooms. NCAM added accessibility features such as captions and audio descriptions to existing 
resources, tagged the resources to indicate which accessibility features they have, created an 
advanced search function that allows users to search for resources with particular accessibility 
features, and developed an accessibility portal that orients users to the adapted site. 

The EDC/Center for Children and Technology served as the external evaluator for the 
User-Centered Digital Library Project. The goal of the evaluation was to inform project activities 
and to document the impact of the adapted version of the digital library on teachers and students 
with disabilities. In Years 3 and 4 of the project, we conducted user testing with teachers and 
students with hearing, vision, and multiple disabilities including mobility impairments, to explore 
the potential impact that the adapted TEACHERS’ DOMAIN site may have on teachers and students 
with hearing, vision, mobility impairments and other disabilities. To better understand how the 
adaptations made on the TEACHERS’ DOMAIN impact users, we compared the use of and outcomes 
for the adapted site to those for the original site. We found a number of important differences: 

Teachers’ Lesson Planning 

There were few qualitative differences in the way teachers searched for resources between the 
original and adapted TEACHERS’ DOMAIN sites. Teachers typically entered the sites by choosing 
topics or conducting keyword searches relating to the objectives they had to cover in their science 
classes, browsing the available resources, previewing them, and tailoring lesson plans to their 
needs. However, the adapted site helped teachers to locate resources more efficiently. Teachers 
used only about half the number of sessions and only about a quarter of the time to select 
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resources on the adapted site compared to the original site. The teachers reported using the acces- 
sibility icons and labels to help them find accessible resources more quickly. 

Teachers’ Classroom Use 

There were a number of differences in how teachers had students interact with the resources 
when using the adapted site compared to the original site. Teachers were less likely to project the 
resources for the whole class and to give students print-outs of the resources, and to have them 
explore the resources in a whole group setting when they used the adapted site. Moreover, teachers 
were more likely to have students explore the resources in small groups. In addition, teachers of 
blind and visually impaired students and teachers of students with multiple disabilities were more 
likely to have students use the resources on computers, and teachers of blind and visually impaired 
students were more likely to have students explore resources individually. 

Students’ Responses to the Teachers’ Domain Resources 

For almost every media type and across all three disabilities groups, students rated the resources 
from the adapted site as easier to use and more likeable. Moreover, students consistently rated 
science lessons that utilized resources from the adapted as more interesting than lessons that 
utilized resources from the original site. 

Student Learning and Understanding of Science Content 

Teachers assessed multiple dimensions of students’ work after each lesson that incorporated 
resources from the TEACHERS’ DOMAIN sites. For most dimensions of student performance, and 
most student groups, students obtained higher ratings for lessons that used resources from the 
adapted site compared to those that used resources from the original site. All student groups 
received higher ratings on engagement with TEACHERS’ DOMAIN resources, quality of work, and 
understanding of science content when using resources from the adapted site. Blind and visually 
impaired students and deaf and hearing impaired students (but not students with multiple 
disabilities) received higher ratings for the level of independence with which they used TEACHERS’ 
DOMAIN resources from the adapted site. Deaf and hearing impaired students and students with 
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multiple disabilities (but not blind and visually impaired students) received higher ratings for 
participation in classroom discussion when using resources from the adapted site. And deaf and 
hearing-impaired students (but not students with multiple disabilities and blind and visually 
impaired students) received higher ratings for participation in small group work, and engagement 
in classroom activities when using resources from the adapted site. 
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Introduction 


The goal of the User-Centered Digital Library Project, conducted by the National Center for 
Accessible Media (NCAM) at WGBH, was to implement IMS/ISO Access Specifications in the 
TEACHERS’ Domain online digital library to enable users (teachers and students with disabilities) 
to conduct searches for accessible content that is formatted to meet specific students’ needs. To 
achieve this goal, NCAM pursued the following activities: 

• Modified the interface of the TEACHERS’ DOMAIN online digital library to embed the capacity 
to customize the presentation of content based on individual user needs (e.g., low vision, non 
visual, visual only, keyboard users) (see Figure 1); 

• Developed preference tools for creating individual user profiles and the sorting mechanisms 
needed to present information appropriately for diverse users with disabilities (see Figure 2); 

• Adapted existing resources to produce one fully accessible curriculum strand in life science 
for grades six through eight; 

• Developed a TEACHERS’ DOMAIN Access Portal to introduce teachers, students, specialists or 
parents to user profiles and the range of accessibility solutions tailored for diverse presenta- 
tions and devices (see Figure 3). 

The EDC/Center for Children and Technology served as the external evaluator for the User-Centered 
Digital Library Project. The goal of the evaluation was to inform project activities in adapting the 
TEACHERS’ Domain digital library and to document the impact of the adapted version of the digital 
library on teachers and students with disabilities. In Years 3 and 4 of the project, we conducted user 
testing with teachers and students with hearing, vision, and physical disabilities to obtain feedback 
on the adapted design for the TEACHERS’ DOMAIN digital library, and data about its potential impact 
on teachers and students. This report summarizes the findings from the user testing. 


1 I am grateful for the contributions to data entry and analysis provided by Ada Uruchima. 
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Methods 


To better understand the potential impact of the adaptations made to the TEACHERS’ DOMAIN site 
we compared teachers’ and students’ use of the original version of the TEACHERS’ DOMAIN site to 
that of the adapted site. The following questions guided our research: 

1. How do teachers use the original and adapted TEACHERS’ DOMAIN sites for their lesson 
planning? 

2. How teachers use resources from the original and adapted TEACHERS’ DOMAIN sites in 
their classrooms? 

3. How do students with visual, hearing and mobility impairments respond to the resources 
from the original and adapted TEACHERS’ DOMAIN sites? 

4. How does the use of resources from the original and adapted TEACHERS’ DOMAIN sites 
impact students’ learning and understanding of science content? 

Figure 1: Sample Search Output for the Adapted Site 
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Figure 2: Accessibility Options in the User Profile of the Adaped Site 
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Figure 3: Accessibility Portal 
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Twelve teachers who worked with students with disabilities participated in the study. The sample 
included four teachers who worked with students with visual impairments, four teachers who 
worked with students with hearing impairments, and four teachers who worked with students 
with disabilities in self-contained special education classrooms that served students with multiple 
disabilities including mobility impairments. Ten of the teachers taught middle school and two of 
them taught high school science classes. 

The teachers were asked to develop six science lessons that incorporate digital resources from 
the life science section of the TEACHERS’ DOMAIN digital library for use by the particular student 
population they served (i.e., students with low vision, no vision, hearing impairment/deafness, 
and mobility impairments). For three of these lessons teachers used the original version of the 
TEACHERS’ Domain site. For another three of the lessons teachers utilized the adapted TEACHERS’ 
DOMAIN site. Teachers then implemented the six lessons in their science classes. A total of 55 
students with disabilities in these teachers’ science classroom had parental permission to partici- 
pate in the research and were included in this study. Sixteen of these students were deaf or hearing 
impaired, 13 were blind or visually impaired, and 26 were students with a range of disabilities 
including mobility impairments. Three of the students were high school students and 52 middle 
school students. 

We used a variety of methods to collect data from teachers and students. Specifically, we used the 
following data sources: 

• Web use data to examine teachers’ use of the two different versions of the TEACHERS’ DOMAIN 
site as they were planning lessons; 

• Lesson plans that the teachers developed; 

• Interviews with teachers to obtain information about their experience planning the lessons 
using the adapted and original sites; 

• Teacher lesson logs to document teachers’ experience with implementing the lessons, the 
usability of the resources, and their assessment of students’ learning and responses; 


UCDL Evaluation Report 


8 


Student surveys conducted after each lesson. The surveys were designed to document students’ 
experience using the digital library, the usability of the resources, and their understanding of the 
science content covered in each lesson. 

Copies of the instruments used are included in Appendix 1. 

Below we report the findings organized around the key research questions. 


Results 

1. How do teachers use the original and adapted Teachers’ Domain sites 
for their lesson planning? 

There were few qualitative differences in the way teachers searched for resources from the original 
and adapted TEACHERS’ DOMAIN sites. Teachers typically entered the sites by choosing topics or 
conducting keyword searches relating to the objectives they had to cover in their science classes, 
browsing the available resources, previewing them, and tailoring lesson plans to their needs, as 
illustrated in the following descriptions offered by two of the teachers: 

“My process of going in and planning was basically the same for both of those, I just went in, 

I knew what objectives or what category or what type of lesson I wanted to address and then 
I went in and just saw what you guys offered and what was available and I knew I wanted to 
do something on DNA because we were talking about the five senses and the color of eyes and 
those types of things and so I just thought, “Well, I’m not planning anything big on DNA." 

So I just went in it first to see what you had and I just love, there were lesson plans that were 
already done and I did modify those and shortened them somewhat but the example that they 
gave with building the DNA with licorice and gumdrops, that was just so huge and so, my kids 
got it which was huge, I mean we could have talked about DNA all day long hut they got that. 
And then the video portions that they had with the definitions, we were able to make defini- 
tion cards off of those and I gave them the printed words but they were able to match pictures 
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and printed words to make their own vocabulary chart off of that so it just really simpli- 
fied things and I didn’t have to recreate the wheel on that which was so huge for me because I 
don’t have hours and hours a day because I have to teach too, that I can just sit and do lesson 
plans so to have that already done, where I could just cut and paste and copy and make my own 
lesson plan out of combinations of plans.” Teacher of students with multiple disabilities 

“What I did is I went by topic mostly, I went through the science and then I went to, depending on 
the unit we were on, I went to the Earth science or whatever unit we were doing at the time and 
then I looked at the sub-categories. I found that the most beneficial way for me to search. I did 
on occasion also go and use the search bar itself where I could type in a topic and hit search but 
the layout; I found it a bit easier. It was so simple to use the topic menus that I decided I might 
miss something if I search for something real specific where it was so easy to use by using just the 
clicking on the menus and the resources and the layout to be able to just scroll through quickly and 
easy. Oh, I could look at that, I could look at that. What I also found myself doing is I’d go through 
and I would save it to my folder even if I had any interest in it and that way I could go back later, 
look in my folder and see what ones were beneficial and what ones I didn’t want to use later.” 
Teacher of visually impaired students 

While the accessibility of a resource was certainly an important consideration for the teachers 
when choosing it, it did not tend to be their primary criterion. For instance, one teacher of 
hearing-impaired students reported that she would use videos that are not captioned and translate 
them into sign language if they were important to make a comparison. 

Regardless of which version of the TEACHERS’ DOMAIN site they were using, virtually all teachers 
reported that they previewed resources to determine their accessibility, and evaluated them using 
criteria that were specific to the needs of their particular student population. A criterion that was 
shared by all teachers was whether the language level of the resources was appropriate for their 
students. 
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“I look at the complexity of it. If there are a lot of scientific terms I tend to shy away from it because 
it’s over their head. Now my visually impaired student, I just always had her listen because she 
couldn’t, there was no way she could see anything so that really didn’t come into play with her. 

The basic thing I look at is, does it apply to what we’re studying and is it something that it’s simple 
enough or can I pause it in between, you know, certain sections and explain it to them that they can 
understand it and sometimes it’s just too technical. If it’s too technical I just don’t even try because it 
just confuses them and it’s not worth the trouble.” Teacher of students with multiple disabilities 

“I’m looking for captions. I look for the level of the language to see if it is not too over their heads. I 
look at the content material to see if it’s interesting or attractive to them, to draw their attention. 
That’s how I was looking. Their language level, of course captions, the pictures and the content.” 
Teacher of students with hearing impairments 

“In general I look to see, first of all, if it’s going to be readable with Jaws and then I look to see if 
for the kids that have some useful vision, if it’s going to be a big enough, clear enough graphic for 
them to see or picture or video or whatever it is, if it can be made big enough and works with any 
magnifying software that the kids might have on there, if the kids are using their own computers. 

And in general, one of the things I look for that’s difficult is language level that works because a 
number for our students have reading levels below grade level. So I also look for readability. Then 
if something is audio-described, there’s so little that’s audio-described that that’s a nice bonus. I 
mean tend to use videos. I use a lot of videos in all my science classes that are audio-described 
videos.” Teacher of students with visual impairments 

Teachers often used the suggested grade range as a way to narrow down their searches. Some 
teachers reported purposefully looking for resources that were targeted to a lower grade range than 
the students they were teaching, because they felt that those resources would be more appropriate 
for them. 
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“First I looked at up through grade 12, 1 mean, since this particular group was supposed to be, or 
two of the students were supposed to be 9 th graders in the fall and one of the students is a senior 
in the fall, but they were all together. I looked and then I found that the stuff that said K-3 was 
better for us, yeah, that it seemed a little bit more within their grasp.” Teacher of students with 
visual impairments 

Even though the teachers were explicitly instructed to set accessibility preferences in their user 
profile when they were using the adapted TEACHERS’ DOMAIN site, only about half of the teachers 
reported making use of this option. 

“I went in and saved certain things to my folder I went through and on each one of those spots it 
actually listed what the adaptation was so I didn’t search as much by the different specific adapta- 
tions, I searched more by content and went through and was able to see what the adaptations 
actually were but that would be an additional thing that you could definitely do is search by that 
way.” Teacher of students with visual impairments 

Teachers’ choice for setting accessibility preferences in their user profile appeared to be related to 
the student population they were serving. All of the teachers of deaf and hearing impaired students 
set accessibility preferences, with the option “captions for video” turned on. However, only two of 
the teachers who worked with visually impaired students, and one of the teachers who worked with 
students with multiple disabilities set accessibility preferences. Teachers who worked with students 
with multiple disabilities indicated that the needs in their groups were so diverse that they needed 
to have all accessibility options turned on anyway. 

One important difference that the teachers noted between the two versions of the TEACHERS’ 
DOMAIN site was that in the adapted site resources were tagged with icons that indicated the acces- 
sibility features they had, and the resources were also labeled “accessible” or “inaccessible” based 
on the preferences set in their user profile. The teachers reported using the icons and labels to help 
them find accessible resources more quickly, as illustrated by the following quotes: 
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“For the unadapted version, of course, I had to play it first and then press a button to the turn the 
captioning on and then if the captioning worked then I knew that it was accessible, but I watched 
all the videos first anyways so I could come up with questions so I knew what I was going to do, 
and then it was nice on the adaptive version because it said that they are closed captioned just by 
looking at it. So it was quicker that way. You didn’t have to watch the videos if you didn’t want to, 

I guess.” Teacher of students with hearing impairments 

“The adapted version was a little bit easier, having a couple more filters that you could use, that 
was certainly beneficial and just to be able to see all those options that are offered, whereas in 
the original version they’re very similar in the way you search and the way you have all the topics 
there but to have those extra pieces that you can narrow down your focus in to even more was 
really beneficial to have. I can see a teacher who is using these for different impaired students to 
be able to just look at those videos that are in their range that have closed captioning or have the 
other different adaptations involved. Very easy.” Teacher of visually impaired students 

The web log data confirms the teachers’ observations. As shown in Table 1, teachers used only 
about half the number of sessions and only about a quarter of the time to locate resources on the 
adapted site compared to the original site. This finding suggests that the information provided in 
the adapted site about the accessibility of resources appeared to have an important effect on the 
efficiency with which teachers were able to locate appropriate resources. 
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Table 1: Average Number of Sessions and Minutes Spent Online Using the Original and 
Adapted Sites 



Teachers of blind and 
visually impaired 
students 

Teachers of deaf and 

hearing-impaired 

students 

Teachers of Students 
with Multiple 
Disabilities, Including 
Mobility Impairments 

All teachers 

Original Site 





Average Number of 
Sessions 

16.00 

16.00 

8.75 

13.67 

Range 

6-26 

6-26 

6-10 

6-26 

Average Number of 
Minutes Spent on the 
Site 

697 

292 

231 

407 

Range 

116-1,912 

275-314 

116-402 

116-1,912 

Adapted Site 





Average Number of 
Sessions 

6.00 

7.00 

3.67 

5.73 

Range 

4-8 

4-10 

1-5 

1-10 

Average Number of 
Minutes Spent on the 
Site 

127 

152 

62 

118 

Range 

31-274 

114-187 

39-79 

31-274 


2. How do teachers use resources from the original and adapted 
Teachers’ Domain sites in their classrooms? 

The teachers integrated resources into lessons on a variety of topics as illustrated in Table 2. The 
lessons had similar topics regardless of the version of the TEACHERS’ DOMAIN site the teachers 
used and the student population served. 

Several of the lessons related to current events (e.g., oil spill, global warming and climate change). 
As one teacher observed, working with a digital library such as TEACHERS’ DOMAIN offers teachers 
the distinct advantage to bring current events to their classroom: 

“...the response was very positive to the resources, you know, and we happened to have some that 
really brought us to where we needed to be when different current events happened this year, 
which was nice, especially the anatomy of the oil spill when, you know because that whole oil spill 
activity. Over this last week, we were able to refer back and bring in that information and then, 
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you know, talk about possibilities, you know, drilling off the east coast and how to write letters to 
senators and stuff." Teacher of visually impaired students 


Table 2: Topics for Lessons in Which the Teachers Integrated TEACHERS’ DOMAIN 
Resources 


Teachers of blind/ 
visually impaired students 

Teachers of deaf/ 
hearing-impaired students 

Teachers of Students 
with Multiple Disabilities 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Comparison of 
Active and Passive 
Transport and the 
Sodium Potassium 
Pump 

Anatomy of an Oil 
Spill 

Animal Hearing 

Adaptation 

Adaptations 

Birth, Growth, 
and Development 

Ecosystems 

Arctic 

Animal Mouth 
Structures 

Animal and Plant 
Cell Comparison 

Cells 

Cells 

Endocytosis and 
Exocytosis 

Cells 

Birds 

Birds' Beaks 

Classifying 

Earth's Simplest 
Organisms 

Ecosystems 

Genes 

Genetics 

Life Cycles 
of Frogs, 
Dragonflies, and 
Butterflies 

Fossils and Ancient 
Humans 

Global Warming 

Global Warming 

Environmental 

Change 

Human Body 
Regulation 

Living and Non- 
Living 

Hurricanes 

Ocean Currents 

Helping Our Earth 

Global Warming and 
the Arctic Ecosystem 

Mouthparts and 
Digestion 

Plant Life Cycles 

Intro to Energy in 
Living Systems— 
autotrophs and 
heterotrophs 

The Importance 
of Oil 

Plant Reproduction 

Needs of Living 
Things 

Weather 

Sexual vs. Asexual 
Reproduction 

Photosynthesis 


Plant Structure and 
Function 

Rattlesnakes 


The Endocrine 
System 

Rainforests 


Plants as Food 
Makers 

Snake Adaptation 


Water 

Tornadoes 


Sea Turtles 

The Horned Lizard 



Weather 


Where Creatures 
Live 

Water and its 
Resources 






Where Creatures 
Live 




The TEACHERS’ Domain sites offer teachers a variety of different media types, including video and 
audio files, text documents, interactive games, images, and student activities. Teachers appreciated 
the ready access to multiple media as the following quote illustrates: 


“And then I had never used technology in my classes unless I went and checked out a video from 
the library but we have the units in our room that can project on the big screen from our computer 


UCDL Evaluation Report 


15 


and so it was so awesome just to have it right at my hand, to not go check out stuff from the 
library, get it back in, preview it. I could just preview it ahead of time and then put the link in a 
special box for me and just click on it and it was up and running for the kids in no time and that 
was absolutely huge and of course with my kids being very visual and having both the video, it’s 
visual and it’s auditory, their comprehension was so much faster and it stuck with them.” Teacher 
of students with multiple disabilities 


Table 3 shows the average number of resources used and media types that teachers selected in 
the different conditions. On average, teachers used a little more than two resources from the 
TEACHERS’ Domain sites per lesson. Teachers of deaf and hearing impaired students used slightly 
more resources per lesson when working with the adapted sites; whereas teachers of blind and 
visually impaired and students and students with multiple disabilities used slightly fewer resources 
when working with the adapted site. 


Table 3: Media Types Used by the Teachers in the Different Conditions 



Teachers of Blind/ 
Visually Impaired 
Students 

Teachers of Deaf/ 
Hearing Impaired 
Students 

Teachers of Students 
with Multiple Disabilities 

All Teachers 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Total 

Number of 
Lessons 

10 

10 

13 

12 

14 

11 

37 

33 

Average 
Number of 
Resources 
Used 

2.20 

2.80 

2.69 

2.42 

2.57 

2.72 

2.49 

2.65 

Video 

13 

16 

23 

19 

16 

13 

52 

48 

Audio 

0 

0 

0 

0 

0 

0 

0 

0 

Interactive 

5 

9 

7 

7 

10 

3 

22 

19 

Image 

0 

2 

5 

1 

5 

3 

10 

6 

Document 

0 

1 

0 

0 

4 

10 

4 

11 

Student 

Activity 

0 

0 

0 

2 

1 

1 

1 

3 

Total 

18 

28 

35 

29 

36 

30 

89 

87 
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The most popular media types across most conditions were videos, followed by interactives. 

Text documents, images, and student activities were used less frequently, and audio files not 
at all. Teachers of blind and visually impaired students decreased their use of all media types 
when working with the adapted versions compared to the original versions. For teachers of deaf 
and hearing impaired students the use of videos and images increased when working with the 
adapted site compared to the original site. Similarly, teachers of students with multiple disabili- 
ties (including those with mobility impairments) increased their use of videos, interactives, and 
images and decreased their use of text documents when working with the adapted compared to the 
original site. 

One reason for the popularity of the videos may be related to their length as noted by one of the 
teachers: 

“It’s a fabulous program I’ve even shared it with some of my colleagues and just having such a 
great library and it’s also great because they’re so short that you can use them real quick in the 
beginning or the end of class to kind of go through a concept. What’s also I think a strong benefit 
to this is a lot of sites have 30 or 40 minute really long videos and it’s really valuable to have these 
short bursts of videos and then you can talk about them and review them so I look forward to keep 
using it.” Teacher of visually impaired students 

Teachers rated the accessibility of the resources they used as fairly high as shown in Table 4. 
Eighty-four percent of the resources from the adapted site and 80 percent of the resources from 
the original site were rated as completely or mostly accessible. 
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Table 4: Teachers’ Ratings of the Accessibility of the Resources Used 



Teachers of Blind/ 
Visually Impaired 
Students 

Teachers of Deaf/ 
Hearing Impaired 
Students 

Teachers of Students 
with Multiple 
Disabilities 

All Teachers 

Accessibility 
of Resources 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Completely 

Accessible 

33% 

22% 

61% 

55% 

93% 

97% 

62% 

58% 

Mostly 

Accessible 

22% 

17% 

31% 

45% 

5% 

3% 

19% 

22% 

Fairly 

Accessible 

28% 

31% 

0% 

0% 

0% 

0% 

9% 

10% 

A Little 
Accessible 

17% 

31% 

0% 

0% 

0% 

0% 

6% 

10% 

Not 

Accessible 
At All 

0% 

0% 

8% 

0% 

2% 

0% 

3% 

0% 


As shown in Table 5, most lessons were conducted in teachers’ classrooms, rather than in a science 
or computer lab. The percentage of lessons implemented in the classroom was slightly lower and 
the percentage implemented in computer labs was slightly higher when teachers worked with the 
adapted site compared to the original site. 


Table 5: Locations Where Teachers Implemented the Lessons 



Teachers of Blind/ 
Visually Impaired 
Students 

Teachers of Deaf/ 
Hearing Impaired 
Students 

Teachers of Students 
with Multiple 
Disabilities 

All Teachers 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Classroom 

30% 

60% 

100% 

100% 

93% 

91% 

78% 

85% 

Science Lab 

40% 

40% 

0% 

0% 

0% 

0% 

11% 

12% 

Computer 

Lab 

30% 

0% 

0% 

0% 

7% 

9% 

11% 

3% 


Teachers incorporated the resources in a variety of ways into their lessons, as shown in Table 6. 
Regardless of which version of the TEACHERS’ DOMAIN sites the teachers were working with and 
the student population they were serving, a large percentage of them used the resources online 
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rather than downloading them. Few teachers in any of the conditions created a slideshow that 
incorporated the resource or had students explore a downloaded resource off line. However, there 
were a number of differences in how teachers had students interact with the resources when using 
the adapted site compared to the original site. Teachers were less likely to project the resources 
for the whole class and to give students print-outs of the resources, and to have them explore the 
resources in a whole group setting when they used the adapted site. Moreover, teachers were more 
likely to have students explore the resources in small groups. In addition, teachers of blind and 
visually impaired students and teachers of students with multiple disabilities were more likely to 
have students use the resources on computers, and teachers of blind and visually impaired students 
were more likely to have students explore resources individually. This pattern of results suggests 
that the resources from the adapted site were used less in a whole-group, teacher-directed manner, 
and students had more opportunities to work with them independently on their own computers 
either individually or in small groups. 
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Table 6: Teacher Use of the TEACHERS’ DOMAIN Resources in the Different Conditions 



Teachers of Blind/ 
Visually Impaired 
Students 

Teachers of Deaf/ 
Hearing Impaired 
Students 

Teachers of 
Students 
with Multiple 
Disabilities 

All Teachers 

Teacher Use of the TD 
Resources 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

1 used the resource(s) directly from 
the Teacher Domain web portal 
(online) 

80% 

80% 

92% 

100% 

86% 

73% 

86% 

84% 

1 downloaded the resource(s) and 
used them off line 

10% 

10% 

0% 

0% 

7% 

18% 

6% 

9% 

1 projected the resource(s) on the 
screen in front of the class 

20% 

30% 

62% 

67% 

57% 

91% 

46% 

63% 

1 printed the resources and handed 
copies to students 

10% 

20% 

8% 

33% 

14% 

27% 

11% 

27% 

1 had students use the resource(s) 
on computers 

80% 

50% 

8% 

8% 

14% 

9% 

34% 

22% 

1 created a slideshow that contained 
the resource(s) and had students 
use the slideshow 

10% 

10% 

0% 

0% 

0% 

0% 

3% 

3% 

1 had students explore the 
resource(s) individually 

40% 

20% 

0% 

0% 

0% 

18% 

13% 

13% 

1 had students explore the 
resource(s) in small groups 

50% 

30% 

8% 

0% 

29% 

18% 

29% 

16% 

1 had students explore the 
resource(s) in a whole group setting 

40% 

60% 

39% 

42% 

36% 

55% 

38% 

52% 

Students explored the reseource(s) 
online using the Teachers Domain 
website 

90% 

60% 

23% 

50% 

43% 

46% 

52% 

52% 

Students explored the resources off 
line using a downloaded version 

20% 

10% 

0% 

0% 

0% 

0% 

7% 

3% 


One of the teachers described the difference in his use of the resources as follows: 


“What I did a little bit differently is when I did the non-adapted versions — it typically was 
something that I projected in front of the whole class whether it was a video or whether it was 
some of the different interactive sites that we use and with the non-adapted ones that’s typically 
the way I used that. When I did the adapted version we typically had them either in small groups 
or one on one with a computer and a set of headphones or things like that or they were actually 
using those, not in front of the whole class, just on their own individually, so that’s one of the 
ways I used them differently. f...]The thing that benefited [the visually impaired students] using 
the adapted versions was having them have the earphones and be able to listen a little bit more. 
There’s no distractions going on around them and they were really able to focus and listen to the 
lessons.” Teacher of visually impaired students 
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The teachers reported that 98 percent of the lessons that utilized resources from the adapted site 
were implemented as planned compared to 88 percent of the lessons that utilized resources from 
the original site. 

The percentage of teachers who reported technical glitches during the implementation of their 
lessons was smaller for the adapted site (51 percent) compared to the original site (61 percent). 
Technical glitches reported included the following: 

• Images for videos and interactives not working 

• Audio descriptions didn’t work 

• Closed captioning didn’t work 

• Problems with opening files 

• Interactives skipping problems 

• Did not have the right version of Shockwave player installed 

• Jaws could not read the writing on a Flash media picture 

Several teachers noted that they continued using the TEACHERS’ DOMAIN digital library beyond 
their participation in this study. 

“It was very simple to get in there as a teacher, the searching was simple, the number of resources 
that you guys have I was surprised at. When I first was looking at this just seeing the variety 
of topics that you guys cover and the number of videos, I’ve continued to use beyond just when I 
used it during those lessons. I just used one today as we were talking about minerals and doing 
minerals in the house so it’s a very user friendly website and just not having to have all those 
bells and whistles to gain access to a lot of the stuff was fantastic.” Teacher of visually impaired 
students 
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3. How do students with visual, hearing and mobility impairments re- 
spond to the resources from the original and adapted Teachers’ Domain 
sites? 


After each lesson, students were asked to rate, on a 5-point scale, how easy it was for them to use 
the TEACHERS’ Domain resources and how much they liked using them (with 1 indicating very 
difficult to use or not at all liking the resource, and 5 indicating very easy to use or very much liking 
the resource). Tables 7 and 8 summarize the results for each of the conditions. For almost every 
media type and every student population, the resources from the adapted site were rated higher 
than the resources from the original site. Across the different student groups, videos were rated as 
most useable, followed by interactives, images, and then text. 


The same pattern of results emerged for students’ ratings of the degree to which they liked using 
the resources. For almost every media type, students’ mean ratings for the resources from the 
adapted site were higher than for those from the original site, indicating that they liked those 
resources more. Students liked videos the most, followed by interactives, images, and text. 


Table 7: Students’ Mean Ratings of the Ease of Use of the Resources 



Blind/Visually Impaired 
Students 

Deaf/Hearing Impaired 
Students 

Students with Multiple 
Disabilities 

All Students 

Ease of 
use of 
Resource 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Video 

4.0 

3.29 

4.44 

3.87 

4.46 

4.30 

4.30 

3.82 

Image 

— 

3.00 

3.50 

3.27 

4.19 

3.62 

3.85 

3.30 

Text 

3.00 

— 

3.56 

3.30 

4.27 

3.77 

3.61 

3.54 

Interactive 

3.33 

3.00 

4.86 

3.73 

4.31 

4.36 

4.17 

3.70 


Table 8: Students’ Mean Ratings of How Much They Liked the Resources 



Blind/Visually Impaired 
Students 

Deaf/Hearing Impaired 
Students 

Students with Multiple 
Disabilities 

All Students 

Like the 
Resource 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Video 

3.77 

3.70 

4.28 

3.43 

4.21 

4.18 

4.09 

3.77 

Image 

— 

3.50 

3.13 

3.55 

4.03 

3.71 

3.58 

3.59 

Text 

2.33 

— 

3.11 

3.11 

3.66 

3.40 

3.03 

3.26 

Interactive 

3.56 

3.00 

3.86 

3.10 

3.94 

4.59 

3.79 

3.56 
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Table 9: Students’ Ratings of How Much They Liked the Science Lesson Overall 



Blind/Visually Impaired 
Students 

Deaf/Hearing Impaired 
Students 

Students with Multiple 
Disabilities 

All Students 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

How 

interesting 
did you 
find the 
lesson 
overall? 

4.46 

3.69 

3.50 

3.18 

4.05 

3.89 

3.95 

3.59 


Students were also asked to rate, using a 5-point scale, the degree to which they found each science 
lesson that utilized resources from the TEACHERS’ DOMAIN sites interesting (with 1 indicating 
not finding it interesting at all and 5 indicating finding the lesson extremely interesting). Table 9 
summarizes the results. Science lessons that utilized resources from the adapted site were consis- 
tently rated as more interesting than lessons that utilized resources from the original site by all 
groups of students. 


4. How does the use of resources from the original and adapted 
Teachers' Domain sites impact students’ learning and understanding of 
science content? 

Both students and teachers were asked to rate the extent to which the resources from the two 
TEACHERS’ Domain sites contributed to students’ science learning. Table 10 shows the percentage 
of students who indicated that the resources contributed to their learning very much. In each 
student group, a larger percentage of students indicated that the resources from the adapted site 
made a significant contribution to their learning than the resources form the original site. 

Teachers’ ratings followed a similar pattern. Fifty-one percent of the teachers reported that the 
resources from the adapted site contributed to students’ science learning very much, whereas only 
42 percent of the teachers felt that the resources from the original site contributed to students 
learning. 
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Table 10: Percent of Students in the Different Conditions Who Indicated that the 
TEACHERS’ Domain Resources Contributed to Their Science Learning Very Much 



Blind/Visually Impaired 
Students 

Deaf/Hearing Impaired 
Students 

Students with Multiple 
Disabilities 

All Students 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Contributed 
to Science 
Learning 
Very Much 

31% 

19% 

61% 

32% 

43% 

44% 

45% 

32% 


Table 11: Percent of Teachers in the Different Conditions Who Indicated that the 


TEACHERS’ Domain Resources Contributed to Students’ Science Learning Very Much 



Blind/Visually Impaired 
Students 

Deaf/Hearing Impaired 
Students 

Students with Multiple 
Disabilities 

Ail Students 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Contributed 
to Science 
Learning 
Very Much 

20% 

30% 

58% 

42% 

77% 

55% 

52% 

42% 


Teachers were also asked to assess a variety of aspects of students’ work after each lesson using 
a five-point scale (with 1 indicating poor and 5 excellent performance). Table 12 summarizes the 
results. Overall, students received medium to high ratings, suggesting that they did well in all of 
the lessons. However, for most dimensions of student performance, and most student groups, 
students obtained higher ratings for lessons that used resources from the adapted site compared 
to those that used resources from the original site. All student groups received higher ratings on 
engagement with TEACHERS’ DOMAIN resources, quality of work, and understanding of science 
content when using resources from the adapted site. Blind and visually impaired students and deaf 
and hearing impaired students (but not students with multiple disabilities) received higher ratings 
for the level of independence with which they used TEACHERS’ DOMAIN resources from the adapted 
site. Deaf and hearing impaired students and students with multiple disabilities (but not blind and 
visually impaired students) received higher ratings for participation in classroom discussion when 
using resources from the adapted site. And deaf and hearing-impaired students (but not students 
with multiple disabilities and blind and visually impaired students) received higher ratings for 
participation in small group work, and engagement in classroom activities when using resources 
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from the adapted site. These findings suggest that while all students benefited form using the 
adapted site, some student subgroups (e.g., deaf and hearing impaired students) benefited more 
than others (e.g., blind and visually disabled students). 

Teachers also commented upon the contribution of the TEACHERS’ DOMAIN resources to student 
learning in the interviews. For instance, one of the teachers described how the use the resources 
fostered students’ understanding and generalization of science concepts: 

“I’m just going to go back and use the DNA example, after we did the DNA we went and did a 
restaurant visit and I said, “O.K. let’s just look at the different jobs. The host would be what 
gender?” and they said male and I said, “O.K. what kind of chromosomes would that person 
have?” and they were like, “Oh, that would beX, Y, blah blah blah.” Which was awesome because 
they really got it and when I crossed it into other areas for generalization they were able to do that 
which is a huge factor with my kids, to generalize to other things outside of that one class, that 
one lesson.” Teacher of students with multiple disabilities 


Table 12: Teachers’ Mean Ratings of Student Work in the Different Conditions 



Blind/Visually 
Impaired Students 
(n=H) 

Deaf/Hearing 
Impaired Students 
(n=13) 

Students with 
Multiple Disabilities 
(n=20) 

All Students 
(n=44) 


Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Adapted 

Original 

Participation in Classroom 
Discussion 

3.11 

3.24 

4.09 

3.69 

4.07 

3.30 

3.75 

3.41 

Participation in Small Group 
Work 

1.36 

2.29 

3.94 

3.41 

2.40 

2.63 

2.57 

2.78 

Engagement in Classroom 
Activities 

3.20 

3.17 

4.22 

3.64 

4.27 

3.71 

3.90 

3.51 

Engagement with Teachers 
Domain Resources 

3.41 

3.00 

4.30 

3.82 

3.78 

3.19 

3.83 

3.34 

Quality of Work 

2.94 

2.84 

4.12 

3.69 

3.93 

3.74 

3.66 

3.42 

Independence in Using the 
Teachers Domain Resources 

1.27 

0.88 

3.68 

3.26 

1.38 

2.10 

2.11 

2.08 

Understanding of the Science 
Content of the Lesson 

3.02 

2.87 

4.05 

3.79 

3.95 

2.90 

3.67 

3.19 
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Summary of Findings and Conclusions 

This study explored the potential impact that the adapted TEACHERS’ DOMAIN site may have on 
teachers and students with hearing, vision, mobility impairments and other disabilities. To better 
understand how the adaptations made on the TEACHERS’ DOMAIN impact users, we compared 
the use of and outcomes for the adapted site to those for the original site. We found a number of 
important differences: 

Teachers’ Lesson Planning 

There were few qualitative differences in the way teachers searched for resources between the 
original and adapted TEACHERS’ DOMAIN. Teachers typically entered the sites by choosing topics 
or conducting keyword searches relating to the objectives they had to cover in their science classes, 
browsing the available resources, previewing them, and tailoring lesson plans to their needs. 
However, the adapted site helped teachers to locate resources more efficiently. Teachers used only 
about half the number of sessions and only about a quarter of the time to select resources on the 
adapted site compared to the original site. The teachers reported using the accessibility icons and 
labels to help them find accessible resources more quickly. While teachers made extensive use 
accessibility icons and labels, only half of the teachers in our sample made use of the user profile 
that lets them search for resources with specific accessibility features, and which could make their 
searchers even more efficient. 

Teachers’ Classroom Use 

There were a number of differences in how teachers had students interact with the resources 
when using the adapted site compared to the original site. Teachers were less likely to project the 
resources for the whole class and to give students print-outs of the resources, and to have them 
explore the resources in a whole group setting when they used the adapted site. Moreover, teachers 
were more likely to have students explore the resources in small groups. In addition, teachers of 
blind and visually impaired students and teachers of students with multiple disabilities were more 
likely to have students use the resources on computers (instead of projecting them), and teachers 


UCDL Evaluation Report 


26 


of blind and visually impaired students were more likely to have students explore resources individ- 
ually. This pattern of results suggests that the resources from the adapted site were used less in a 
whole-group, teacher-directed manner, and students had more opportunities to work with them 
independently on their own computers either individually or in small groups. 

Students’ Responses to the Teachers’ Domain Resources 

For almost every media type and across all three disabilities groups, students rated the resources 
from the adapted site as easier to use and more likeable. Across the different student groups, 
videos were rated as most useable and likeable, followed by interactives, images, and then text. 
Moreover, students consistently rated science lessons that utilized resources from the adapted as 
more interesting than lessons that utilized resources from the original site. These findings suggests 
that the increased accessibility of the resources in the adapted site makes it easier for students 
to use these resources and more likely that they like using them and find their science content 
appealing. 

Student Learning and Understanding of Science Content 

Teachers assessed multiple dimensions of students’ work after each lesson that incorporated 
resources from the TEACHERS’ DOMAIN sites. For most dimensions of student performance, and 
most student groups, students obtained higher ratings for lessons that used resources from the 
adapted site compared to those that used resources from the original site. All student groups 
received higher ratings on engagement with TEACHERS’ DOMAIN resources, quality of work, and 
understanding of science content when using resources from the adapted site. Blind and visually 
impaired students and deaf and hearing impaired students (but not students with multiple 
disabilities) received higher ratings for the level of independence with which they used TEACHERS’ 
DOMAIN resources from the adapted site. Deaf and hearing impaired students and students with 
multiple disabilities (but not blind and visually impaired students) received higher ratings for 
participation in classroom discussion when using resources from the adapted site. And deaf and 
hearing-impaired students (but not students with multiple disabilities and blind and visually 
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impaired students) received higher ratings for participation in small group work, and engagement 
in classroom activities when using resources from the adapted site. These findings suggest that 
while all students benefited form using the adapted site, some student subgroups (e.g., deaf and 
hearing impaired students) may have benefited more than others (e.g., blind and visually disabled 
students). 

This study provides initial evidence of the potential impact of the adapted TEACHERS’ DOMAIN site 
on teachers’ lesson planning, classroom practices, and student learning. Given its limited scope, it 
was not designed to investigate causal claims about the effects of the adapted site. However, the 
findings reported here are consistent with such causal connections and suggest hypotheses that 
could be explored in larger-scale experimental studies. 
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